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WITH INFINITE HORIZON 



SVEN HAADEM, BERNT 0KSENDAL, AND FRANK PROSKE 



Abstract. We prove maximum principles for the problem of optimal control 
for a jump diffusion with infinite horizon and partial information. The results 
are applied to partial information optimal consumption and portfolio problems 
in infinite horizon. 



1. Introduction 
In this paper we consider a control problem for a performance functional 



J(u) = E 



n 



where X(t) is a controlled jump diffusion and u(t) is the control process. We allow 
for the case where the controller only has access to partial-information. Thus, we 
have a infinite horizon problem with partial information. Infinite-horizon optimal 
control problems arise in many fields of economics, in particular in models of eco- 
nomic growth. Note that because of the general nature of the partial information 
filtration £ tl we cannot use dynamic programming and Hamilton- Jacobi-Bcllman 
(HJB) equations to solve the optimization problem. Thus our problem is different 
from partial observation control problems. 

In the deterministic case the maximum principle by Pontryagin (1962) has been 
extended to infinite-horizon problems, but transversality conditions have not been 
given in gerneral. The 'natural' transversality condition in the infinite case would 
be a zero limit condition, meaning in the economic sense that one more unit of good 
at the limit gives no additional value. But this property is not necessarily verified. 
In fact [U provides a counterexample for a 'natural' extension of the finite-horizon 
transversality conditions. Thus some care is needed in the infinite horizon case. 

There have been a variety of articles on infinite-horizon problems. E.g. in [6] 
it is stated a 'natural' extension to infinite horizon discounted control problems. 

We refer to |13) for more information about stochastic control in jump diffusion 
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markets, to [8] for a background on infinite-horizon backward stochastic differential 
equations and [IT| for a general introduction to infinite-horizon control problems in 
a deterministic environment. 

In this paper we prove several maximum principles for an infinite horizon opti- 
mal control problem with partial information. The paper is structured as follows: 
In Section 4 we prove a maximum principle version of sufficient type (a verification 
theorem). In section 5 we give some examples, before we prove a (weak) version of 
a necessary type of the maximum principle in section 6. 

In a forthcomming paper pQ, the case of infinite horizon for delay equations is 
treated. 

2. Preliminaries 

Let B(t) = B(t,u) = (Bi(t,u)),...,B n (t,u)), t > 0, lu e £1 and N(dz,dt) = 
N(dz,dt) — v{dz)dt = {N\(dz^ dt), . . . , N n (dz, dt)) be a n-dimensional Brownian 
motion and n independent compensated Poisson random measures, respectively, on 
a filtered probability space (fi, T, {Tt}t>o, P)- Let X(t) = X u (t) be a controlled 
jump diffusion, described by the stochastic differential equation 

dX(t) = b(t,X(t),u(t),u)dt + a(t,X(t),u(t),uj)dB(t) 
(1) +/ 6(t,X(t),u{t),z,u;)N(dz,dt);Q < t < oo 

X(0) = x e R™, 

where b : [0, oo] x M™ x U x O -> R" is adapted, a : [0, oo] x R" x U x n ->• K" xn is 
adapted and 9 : [0, oo] x R" x U x D, ->• R" x ™ is predictable (see [§]). See e.g. 0, 
|13| for notation and more information. Let 

be a given subfiltration, representing the information available to the controller at 
time t;t > 0. The process u(t) is our control, assumed to be {£t}t>o predicatble 
and with values in a set U C R™. Let As be our family of Et -predicatble controls. 
Let TZ denote the set of functions r : [0, oo] x MJJ ->• R™ x ™ such that 



|^ij(t, a:, u, z)rij{t 1 z)\vj(dz) < oo for all i, j, t, x. 
Let / : [0, oo] x R" x U x fl — > R™ be adapted and assume that 



E 



\f{t,X(t),u{t)^)\dt 



n 



< oo for all u G 



Then we define 



J{u) = J5 



f(t,X(t),u(t),oj)dt 







to be our performance functional. We study the problem to find u € As such that 

(2) J(u) = sup J(u). 

ueA e 
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Let us define the Hamiltonian H : [0, T] x R" x U x R n x W ixn x K -> R, by 

-ff (£, x, q, r) = f{t,x,u,w) + b T (t, x, u : w)p + tr(a T (t, x, u, ui)q) 
n * 

(3) + X! / ^i.j(t,x,u, z,uj)r l , J (t,z)iy j (dz). 

ij=i J K 

For notational convenience we will in the rest of the paper suppress any u) from the 
notation. The adjoint equation in the unknown J^-predictable processes (p(t), q{t), r{t. 
is the following 

dp(t) = -V x H(t, X(t),u(t),p(t), g(t),r(t, -))dt + q{t)dB{t) 

(4) + / r(t,z)N(dz,dt). 

3. Existence and Uniqueness 

In this section we prove a result about existence and uniqueness of the solution 
(Y(t),Z(t),K(t, 0) of infinite horizon BSDEs of the form; 

dY{t) = -g(t, Y(t), Z(t), K(t, -))dt + Z{t)dB(t) 

(5) + / K{t, QN[d(, dt); < t < t, 

(6) limK(*)=C(r)l [0 ,oo)(T), 

where r < oo is a given J^-stopping time, possibly infinite. Our result is an 
extension to jumps of Theorem 4.1 in [7], Theorem 4 in [S] and Theorem 3.1 in |15| . 
It is also an extension to infinite horizon of Theorem Lemma 2.1 in [5], See also 
[II], [TU], [5] and [12]. We assume the following: 

(1) The function g : O x R+ x R fc x R kxd x TZ -)• R k is such that there exist 
real numbers /x, A, Ki and K 2 , such that Ki,K 2 > and 

(7) A > 2n + Kl+K%. 

We assume that the function g satisfies the following requirements: 
(a) g{-, y, z, k) is progessively measurable for all y, z, k, and 

\g(t, y, z, - g(t, y, z', k'(-))\ < K x \\z - z'\\ 

(8) +K 2 \\k(-)-k'(-)\\ R , 
where 

\\K)\\r= I k 2 (CMd(), 



and ||z|| = [Tr(zz*)]i. 

(b) 

(9) (y - y', g(t, y, z, k) - g(t, y', z, k)) < fi\y - y'\ 2 
for all y, y' , z, k a.s. 

(c) 

(10) El e xt \g(t, 0, 0, 0)\ 2 dt < oo. 

Jo 



4 



SVEN HAADEM, BERNT 0KSENDAL, AND FRANK PROSKE 



(d) Finaly we require that 

(11) y g(t,y,z,k), 

is continuous for all t, z, k a.s. 
(2) We have a final condition £, which is Jv-measurable such that 
E(e Xr \£\ 2 ) < oo and 



(12) 



(13) 



E / e M \g{t^ um ^ t )\ z dt< 



where £f = £7(£|.Ft) and 77,^ are s.t. 



i = EH+ r,( a )dB 



i>(s,ON(d(,ds). 



A solution of the BSDE (JS])-©, is a trippel (Y t , Z t , K t ) of progressively measurable 
processes with values in i x 1 x 1 s.t. Z t , K t — when t > t, 

(1) S[sup e ^|r t | 2 + / T e ^|Z s | 2 d S + /;/ R „e As X 2 ( S ,CMrfCKs] <oo, 

t>0 

(2) Y t = Y TAt + ffs ds - Z s dB s - j^J J K K(s, ()N(dt, ds) for all 
deterministic T < 00 and 

(3) Y t = £ on the set {t > t}. 

Remark 3.1 (Infinite Horizon). This incorperates the case where t(u) = 00 on some 
set A with P{A) > 0, possibly P(A) = 1. 

Theorem 3.1 (Existence and uniqueness). Under the above conditions there exists 
a unique solution (Y t , Z t , K t ) of the BSDE ((5])-((6]), which satisfies the condition; 



K 2 (s,CHdQds] 



E[ sup e At |Y t | 2 + [ e Xs (\Y s \ 2 + \\ Z s \\ 2 )ds + f e ; 

0<t<r Jo JO 

(14) < c£[e A ^| 2 + f T e Xs \g( Sl 0, 0, 0)| 2 d S ], 
Jo 

for some positive number c. 

Proof. First, let us show uniqueness. Let (Y, Z, K) and (Y', Z', K') be two solutions 
satisfying ^ and let (Y, Z, i?) = (Y - Y', Z - Z',K - K'). From Ito's Lemma 
we have that 



„A*Atit> i2 



TAt 



e As (A|Y s | 2 + ||Z s || 2 ) + e > 

■TAr 



< e As I Yr 1 2 + 2 



t At 



e^MY.r + ITilY.I x ||Z fl 



+ K 2 |Y s |e As (/^ 2 ( S ,C)^(dC)) ! 



ds 



TAt 



e As (Y s ,Z s d£g 



*At 
TAt 



(At 



..As 



[K 2 (s,()+2K(sX)Y(s)] N(d(,ds). 
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Combining the above with the fact that 2ab < a 2 + b 2 we deduce since A > 2/j 
Kf + K%, that for t < T 

E[e xtAT \Y tAT \ 2 } < E[e XTAT \Y T \ 2 ] 
the same holds with A replaced by A , with A > A' > 2/j + K 2 + K 2 



E 



Ai At 



YtAr 



< e^- x '^E 



XT At I \/- |2-i 

e \Y T \ !{T<r} 



Condition (fTl| implies that the second factor on the right hand side remains 
bounded as T — > oo, while the first factor tends to 0. This proves uniqueness. 



Proof of existence. For each n G N we construct a solution (Y",Z",K") of the 
BSDE 



Y? = £ + / g(s, Y™, Z?, K^)ds - / ZyB. 



tAT 



K n ( S ,C)N(d(,ds) 



by letting {(Y~™, Z™, if™); < t < n} be defined as a solution of the following BSDE: 



K n (s,0N(d(,ds) 



Y t n = E[^\T n }+ J l l0 , T] (s)g( 3 , Y s n ,Z%, K^)ds- J Z n s dB. 
for < t < n and {(Y t n , Z?, K?)\ t > n} defined by 

Z? = Vt, 



and 



for t > n. Next, we find some a priori estimates for the sequence (Y n , Z n , K n ). For 
any e > 0, p < 1 and a we have for all f > 0, y G K fc , z e M fcxd , fc G K with c = ±, 

2(y, ff(f, y, z, k)) = 2(y, g(t, y, z, k) - g(t, 0, z, k)) 
+ 2(y, g(t, 0, z, k) - g(t, 0, 0, 0)) + 2(y, g(t, 0, 0, 0)) 

< (2/i + iif? + + e)M 2 + p || 2 f +a / k 2 ((Md() 

+ c|.g(i,0,0,0)| 2 . 
From Ito's Lemma we have 



e XtAT \YJiJ 2 



e Xs (X\Y s n \ 2 +p\\ Z n s || 2 ) + a e Xs (K ") 2 ( S , Qu(dQ 



ds 



< e As |?7| 2 + c / e Xs \g(s,0,0,0)\ 2 ds 

J tAT 



e AS < Y™.Z n s dB s > 



[(K n ) 2 (s, C) + 2K n (s, C)Y n (s)] N(d(, ds), 
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1 K \ - ±K\ 

p 1 a ^ 

and the matingale inequality it follows that 



with A = A - 2yu - -K{ - ±K\ - e > 0, p = I - p > Q and a = 1 - a. From this 



E 



sup e^\Y t l r \ 2 

t>s 



e Xr {\Y?\ 2 + || Z: f ) + / 



e |£| + / e Ar |ff(r,0,0,0)| 2 dr 



Let to > n and define AY t := Y t m - Y t n , AZ t := Z t m - Z t " and Aif f := K r t n - K?, 
so that for n < t < to, 



/' 



g(s,Y s m ,Z™,K™)ds- AZ s dB, 



ItAT 

It then follows that 



/■mAr pmAr p 

/ AZ s dB s - / / 



A J ftT(s,C)iV(dC,rfs)- 



Ai At 



A1W| 2 + r Ar {e As (A|AF s | 2 + || AZ S f) + e Xs f (AK) 2 ( S ,0v(d()}ds 

JtAr L ^Eq J 

/•mAr 

= / e Xs (AY s ,g(s,Y s m ,Z™,K™))ds 

JtAr 

-2 / e Xs (AY s ,AZ s dB s ) 

JtAr 

[(AK) 2 (s, C) + 2AK(s, C)AF(s)] iV«, d«) 



pmAr pmAr 



■i 

Jti 



2<e As \ V \'c e As | 5 (s,0, 0, 0)|^ S -2 / e As {AY S , AZ s dB s ) 



pmA 
JtAr 



pmAr p 

- e Xs [(AK) 2 ( S ,0 + 2AK( S ,()AY( S )]N(d(,d S ). 

JtAr J Kg 

From the same arguments as above 



E 



sup e AtAT |AF tAT | 

n<t<m 



+ 



f mAT \e Xs (\AY s \ 2 + || AZ S \\ 2 ) + e Xs f (AK) 2 ( S ,(MdC)}d S 

JnAr L JR™ ' 

<1E\[ e Xs \g(s,tv,n 2 ds . 

.JnAr 

The last term in the above equation goes to zero as n — > oo. Now, for t < n 
AY t = AY n + / Us, Y s m , Z™, KT) - g(s, Y?, Z?,K?)\ds - / AZ s dB s 

JtAr 1 J JtAr 

pnAr p 

- \ \ AK(s,0N(d(,ds). 
Using the same argument as in the case of uniqueness, we have that 

E[e xt ^\AY tAT \ 2 } < E[e xtAT \AY n \ 2 ] < ce\ f e Xs \g( S ,^r, s ^ s )\ 2 d S . 
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It now follows that the sequence (Y n , Z n ,K n ) is Cauchy in the norm 

\\(Y,Z,K)\\ :=E[ sup e xt \Y t \ 2 + [ e Xs {\Y s \ 2 + \\ Z s \\ 2 )ds 
0<t<r Jo 

K 2 (s,CMd0ds}. 



So, we have that there is an unique solution to the BSDE (0-([6]), which satisfies 
for all A > 2/i + K\ + K 2 , the condition 



E 



sup e xt \Y t \ 2 + [ T e Xs (\Y s \ 2 + || Z s f)d S + [ T e Xs [ 
o<t<r Jo Jo Jm% 

e xr \e+ [ T < 

Jo 



e As | 5 (s, 0,0,0)|^s 



□ 



4. Optimal control with partial information and infinite horizon 
Now, let us get back to the problem of maximizing the performance functional 



J(u) = E 



f(t,X(t),u(t))dt 



where X(t) is of the form ([T]). Our aim is to find a u £ As such that 

J(u) = sup J(u), 

where u(t) is our previsible control adapted to a subfiltration 

£t c F t , 

with values in a set U C W 1 . Let H be the Hamiltonian defined by and p 
the solution to the adjoint equation (U). Then we have the following maximum 
principle; 

Theorem 4.1 (Sufficient Infinite Horizon Maximum Principle). Let u G As and 

let (p(t), q(t),f(t, z)) be an associated solution to the equation (j4]). Assume that for 
all u £ As the following terminal condition holds: 

J^[p(t) T (X(t)-X(t))} 



(15) 



< E 



< oo. 



Moreover, assume that H(t,x,u,p(t),q(t),f(t,-)) is concave in x and u and 



E 



maxE 



(16) 

In addition we assume that 
(17) E / (X(t)-X-(t)f[qq T 



H(t,X(t),u{t),p(t)^(t),r(t,-))\£t 
H{t,X(t),u,p{t)A{t),f{t,-))\£ t 



ff L (t, z)v{dz)]{X{t) - X u {t))dt 



(18) E 



p{t) 1 [o-a 1 (t 1 X{t),u{t))+ / 66 1 (t,X(t),u{t))v(dz)]p(t)dt 



< CO, 



< oo, 



(19) E \V u H(t,X(t),u(t),P(t),Q(t),nt,-))\ 



< CO, 



cS 
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and that 
(20) 



E 



\H(s,X(s),u(s),p(s),q(s),f(s,-))\ 



< oo 



for all u. 

Then we have that u(t) is optimal. 

Remark 4.1. Note that, since p(t) has the economic interpretation as the mar- 
ginal value of the resource (alternativly the shadow price if representing an outside 
resource), the requirement 



< E 



t— > oo 



< OO, 



has the economic interpretation that if the marginal value is positive at infinity we 
want to have as little resources left as possible. 

Remark 4.2. The requirement in the finite horizon case that p(T) — does not 
translate into limp(T) = as was shown in the deterministic case in [¥]. 

T^oo 

Proof. Let I°° := E[f™ (f(t, X(t),u(t)) - f(t,X(t),u(t)))dt] = J(u) - J(u). Then 

joo = joo _ joo _ joo _ jo 0) where 



7f° := E 



(H(s, X(s),u(s),p(s), q(s), f (s, •)) 



H(t,X(s),n(t),p(s),q(s),f(s,-)))ds 



E 
E 



p(s) T (b{s,X(s),u(s)) - b(s,X(s),u(s)))ds 



tr[q(s) T (o-(s,X(s), u{s)) - a(s, X(s), u(s)))]ds 



and 



IT ■= E 



/°°E / i6{s,X{s),u{sl 
Jo „■ „■ Jk 



z) 



i,3 o 



— 9(s, X(s), u(s), z)) T fi_j (s, z)vj(dz)ds 
We have from concavity that 

(21) H(t, X(t), u(t),p(t), q(t),r(t, •)) - H(t, X(t),u(t),p(t),q(t),r(t, •)) 

< V x H(t, X(t), u{t),p{t)A{t),r(t, -)) T (X(t) - X(t)) 

(22) + V u H(t, X(t), u(t),p(t), q(t),r(t, -)) T (u(t) - u(t)). 
Then we have from (fTTH) . (TIT)]) and that u(t) is adapted to £t, 



> V U E 



H(t,X(t),u,p(t),q(t),r(t,-))\£ t (u(t)-u(t)) 

. U — u{t) 



(23) 



= E 



V u H{t, X(t),u(t),p(t),q(t), f(t, -)) T (u(t) - u(t))\£ t 
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Combining (j4j). ijTTL (|2Tjl. (|22f and ([23]) 



If < E 
= E 



V x H(t, X(s), U(s),p(s), q(s), f(s, -)) T (X(s) - X(s))ds 



(X( S )-X( S )) T dp(s) 



Now, using fll"5f and Ito's formula 



< E 

= E 



\im[p(t) T (X(t)-X(t))} 

t— too 



lim 



p(s) T (b(s, X(s), u(s)) - b(s, X(s), u(s)))ds 



+ / p(s) T (a(s,X(s),u(s)) - a(s,X(s),u(s)))dB(s) 



+ / p(sy (6(s,X(s),u(s),z) -9(s,X(s),u(s),z))N(dz,ds) 



+ / (X(s) - X(s)) T (-V x H(s,X( S ),u( S ),p(s),q(s),f(s,-)))d S 



+ / q(s) T (X(s) - X(s))dB(s) 



+ / / r(s,z)(X(s)-X(s))N(dz,ds) 



+ f tr[q( S ) T (a(s,X( S ),u( S )) - *(s,X(s),u(s))) 



ds 



+ (0(s,X(s),u(s),z)-e(s,X(s),u(s),z)) T hj(s,z)iy j (dz)ds 



+ / / (6(s,X(s),u(s),z) -9(s,X(s),u{s),z)) T r(s,z)N(dz,ds) 
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From (QUI, (jig), we have that 



< E 







lim 


11 


t— >oo 





p(s) T (b(s, X(s),u(s)) - b(s, X(s),u(s)))ds 



+ / (X(s) - X(s)) 1 (-V x H(s,X(s),u{s),p(s),q(s),r(s,-)))ds 
Jo 

rt 



+ / tr 
'o 

ft 



ds 



[q(sf(a(s 7 X(s),u(s)) - &(s, X(s), «(«))) 
+ I y2 [ (o(s,X(s),u(s),z) — §(s,X(s),u(s),z)\ fi y j(s,z)vj(dz)di 

.In . . ./]Rg ^ / 

p(s) T (b(s, X(s), «(*)) - &(*, X(s), u(«)))ds 



= £ 



(X(s) - X(s)f(-V x H(s, X(s), u(s),p(s), q(s), r(s, -)))ds 



+ r tr [q(s) T (a(s, X(s), u(s)) - a(s, X(s),u(s))) 



ds 



^2 / {9{s,X(s),u(s),z) -9{s,X{s),u{s),z)) T f l ^(s,z)v j {dz)di 



TOO I TOO I roc I roc 

— 1 1.2 + J l "+" 1 1,3 + 1 1A- 

Finally, combining the above we get 

J(u) - J(u) < i? - it - IT - 1 



oo 



^ TOO TOO TOO TOO 

— J l 1 2 2 3 1 i 



< 0. 



3 J 4 
'.3 — 1 i 



This holds for all u G As , so the proof is complete. 

5. Examples 

Example 5.1 (Optimal Consumption Rate Part I). Let 



J(u) = E 



e~ pt In (u(t)X(t))dt 



where 

dX(t) = X(t)(p(t) - u{t))dt + X(t)a(t)dB(t), 
X(0) = x o , 
and p > 0. We have that 

1 



X(t) = X exp 



2/ 



Ms) - u(s)) - -a 2 (s)}ds + / a(s)dB(s) 
z Jo 



□ 



Then we deal with the problem of maximizing J(u) over all u(t) > 0. We have the 
Hamiliton function takes the form 

H(t, x, u, p, q) — e~ pt ln(ux) + x(/i — u)p + xaq, 
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so that we get the partial derivatives 



and 



This gives us that 



V x H(t,x,u,p,q) 



V u H(t, x, u,p, q) 



-pi 



+ (n - u)p + aq, 



-pi 



xp, 



- d P {t) = 



-pi 



X(t) 



+ (p(t) - u(t))p{t) + a(t)q(t) 



dt-q(t)dB(t). 



so that 



u(f) 



- P t 



X(t)p(t) 



Let us try the infinite horizon BSDE with terminal condition \\mp{t) = ? 

t— »-oo 



(24) - dp(t) = 

(25) limp(t) = 



-pi 



X(t) 



+ (p{t) - u{t))p(t) + a(t)q(t) 



dt- q(t)dB(t), 



Lemma 5.1 (Solution of infinite horizon linear BSDE with jumps). Let A(t),(3(t) 
and a(t, £) be Tt -predictable processes such that 



E 



< oo, 



{\A(t)\+p 2 (t)+ / a 2 (sXHdO}dt 
and define T t ^ s as the solution of the linear SDE 

dT t , a =T t -J A(t)dt + /3(t)dB(t) + a(t, ()N(d(, dt)),s>t>0, 



r M = 1. 

Let C(t) be a predictable process such that 



E 



T^ s \C(s)\ds 



< oo. 



Then a solution (Y(t), Z(t), K(t,£)) of the linear BSDE 



A(t)Y(t) + Z(t)(3(t)+C(t)+ / a(t,OK(t,()dv(C) 



-dY(t) 



- Z(t)dB(t) - I K(t, C)N(d(, dt), 

limY(t) = 0,t -> oo, 
is given by 

Y(t) = E 

If in additon 



dt 



/OO 
T t:S C{s)ds\F t 



,t>0. 



E 



e M \Y{t)\ 2 dt 



Uo 



< oo, 



where X as in ([7]), then Y{t) is the unique solution. 
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Proof. By Ito's Lemma we have that 

d(T 0tt Y t ) = -T 0tt C t dt + r , t (Z t + Y t p t )dB t 

Y(t)a(t, QT 0<t + K(t, QT 0<t + K(t, ()a(t, C)r 0)t l N(d(, dt). 



So 



/oo 
T 0>s {Z s +Y s p s )dB{s) 

Y( s )a(s, c)r , s + K( S , c)r , s + K( S , c)a(«, C)r , s l N(dC, ds). 



To,tY t 



By taking expectation we get the desired result. The uniqueness follows from 
Theorem 3.1. □ 

From the above lemma we see that the solution of the linear, infinite horizon 
BSDE fH) - AH]) is 

f , e-P s 



p{t) = E 



-ds IF* 



where 

Hence 

and 
So 



r t = e 



f*[Ms)-u(s))-^ 2 (s)]ds+^a(s)dB(s) = X W 



;r 



p(t) = -e- pt J—. 
P X(t) 



limp(t)(X(t) - X(t)) > \im p(t)X(t) > 0. 

t—>-oo t—too 



u(t) = p, 

is an optimal control. 

Example 5.2 (Optimal Consumption Rate - part II). Let 



J(u) = E 



e' pt In (u(t)X(t))dt 



where 

dX(t) = X(t)n(t)(l - u{t))dt + X(t)a(t)(l - u{t))dB{t), 
X(0)=x o , 
and p > 0. We have that 



X(t) = X exp 



{H( s )(l-u(s))-^(s)(l-u(t)f}ds 



+ / <r(s)(l -u(s))dB(s) 



Then we deal with the problem of maximizing J(u) over all u(t) > 0. We have the 
Hamiliton function takes the form 

H(t, x, u,p, q) — e~ pt \n(ux) + x/j,(l — u)p + xa(l — u)q, 
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so that we get the partial derivatives 



-ft 



V x H(t,x,u,p,q) = h/i(l -u)p + a(l - u)q, 



and 



-ft 



\7 u H(t, x,u,p,q) = xpp — xaq. 



This gives us that 



dp{t) = 



,—pt 



X(t) 



+ - u(t))p(t) + a(t)(l - u{s))q(t) 



dt- q(t)dB(t). 



So that 



u(t) 



-pt 



X(t)([ip(t)+aq(t)) 
Let us try the infinite horizon BSDE with terminal condition 



lim p(t) = 0, so that 

t— »oc 



-dp(t) 



-pi 



(26) 



X(t) 
- q{t)dB{t) 



+ /i(t)(l - u{t))p(t) + a{t)(l - u{s))q(t) 



dt 



(27) limp(*)=0. 

t— >oo 

From the above lemma we see that the solution of the linear, infinite horizon BSDE 

mi - dsn is 



p(t) = E 



r , s e-p* 



ds\Ft 



where 



r t = exp 



M(s)( l_ u(s ))__ a 2 (s)(1 _ u(s)) 2 



ds 



a(s)(l - u{s))dB{s) 
X(t) 



•J'o 



Hence 

and 
Since 



p(t) = l e ~ pt J—. 
P X(t) 



limp(t)(X(t) - X(t)) > lim p(t)X(t) > 0. 

t—toc t—too 



,-pt. 



X(t) 



X 



X(t) 



(n(t) - u{t))dt 



^^/m + e-^a { t) d B { t), 



we must have that 



P X(t) 



a(t). 
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So 



U (t) 



fi + a 



is an optimal control. 

Example 5.3 (Optimal consumption rate - part III). As above, let 

" />oo 

J(u)=E / e- pt \n(u(t)X(t))dt . 
Jo 

But add a jump part 

dX(t) = X{t)(n{t)-u{t))dt + X(t)<r{t)dB(t)+X(t) / 9(t)zN(dz,dt) 

JR 

X(0) = so, 

and we also add the assumption that we only know a subset of the information given 
by the market available at time t, represented by E% <Z J~t- Let p > 0, be a random 
variable adapted to J- t . Then we deal with the problem of maximizing J(u) over all 
u(t) > 0. We have 

H(t,x,u,p,q,r) = e~ pt \n(ux) + x(/i — u)p + xaq + x / 9(t)zr(t, z)v{dz) 



V x H(t,x,u,p,q,r) 



-pt 



+ (fi — u)p + aq + / 9(t)zr(t,z)v{dz), 



V u H(t,x,u,p,q,r) 



-pi 



xp 



and 



,-pt 



-dp{t) = [— + (At(t) - u{t))p(t) + a(t)q(t) + I 0(t)zr(t, z)v{dz)]dt 
-q(t)dB(t)- I 9{t)zN(dz,dt), 



limp(t) = 0. 

t— >oo 

If we maximice 

E[H(t,X(t),u,p(t),q(t),f(t,-))\£ t ], 

we get that 

V u E[H(t,X(t),u,p(t),q(t))\£t] = E[V u H(t,X(t),u,p(t),q(t))\£ t ] 



-pi 



x(t)p(t)\£t] 



So that 



u(i) =£[_ 



-pt 



X{t)p{t) 

The solution of the linear, infinite horizon BSDE (I24[) - (|25p is (see |10j ) 



ffl = E 



f t X., 



ds\E t 
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where 

dT t =X(t)(fj,(t)-u(t))dt + X(t)a(t)dB(t)+X(t-) [ 9(t)zN(dz,dt), 
X(0) = 1. 

So 

f _ X{t) 
1 1 — • 

x 

Hence 

p(t) = -X--e~' }t = -e-^ — 



x T t P P X(t) 

Therefore we have that 



\imp(t)(X(t) - X{t)) = \\m.p{t)X{t) > 0. 

So 

u(t) = E[p, \St] 

is an optimal control. 

Example 5.4 (Optimal Portfolio Selection With Consumption). For this example 
let us look at a market with two investment possibilities: 

(1) A bond or bank account 

dZ (t) = pZ (t)dt. 

(2) A stock 

dZ x (t) = [iZ x (t)dt + oZ x (t)dB(t). 

Let (Yoi Y\) denote the amount the agent has invested in the bonds and stocks repec- 
tively at time t. Consider then u(t, u>) = u(t), the fraction of the wealth invested in 
the stocks, e.g. 

z i(t) 

U{t) ~ Z (t) + Zl (t)- 
Further let \(t,u) = X(t) be the consumption rate relative to the wealth so that the 
investor controls 

c(t) = («(*), A(t)). 

Then let 

J x ' u (s,z) = E 3 ' 



°° c _ 5(s+t) (\(t)X(t)r 



1 

be a performance functional, where 

dX(t) = X(t) [(p + u{t)(p - p) - \{t))dt + <7u(t)dB(t)} , 
and p>0. We have that 

X(t)=xoCxp I [p + u(s)(p — p) — \(s) — \ ) cr 2 u 2 ]ds + / o-u(s)dB(s) . 

Then we want to maximize J u ' X (s,t) over all I = (u(t), A(t)) 7 A > 0. We have that 

H{t, x,l,p,q) = e -^+') (A(t)X(t))7 + x(p + u(s)(p - p) - X)p + xauq, 

7 

so that 

V x H{t, x, l,p, q) = e-^+'U^- 1 + (p + u(p -p)- \)p + ouq. 
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Further, we also have 

-dp(t) = [e _4 < s+t >A 7 (t).X" 1, - 1 (t) + (p + u(i)(/i -p)- \(t))p + cru(t)q}dt - qdB(t). 
and 

V u H(t, x, l,p, q) = (fi - p)xp + xaq, 
V X H(t,x,l,p,q) = e- 5 ^(\(t)y- 1 X-' - xp. 

So that 

m = - { -^ P (t), 

a 

and 

~ 1 _J_ 6(. + t) 

X = —pi- 1 e t- 1 . 
x 

Then 

S(s + t) 1 1 1 5(s + t) 

dp(t) = -e i-i — pT-i (t)dt - [p + u(i)(fj, - p) - — (t)p^- 1 e i' 1 ]p(t)dt 
X X 

+ au{t) ^~ P \ {t)dt - (M - p) p(t)dB(t) 
a a 

= -pp(t)dt - P \ {t)dB{t). 
a 

So to ensure that the requirement 
is satisfied we need that 



E[\imp(t)(X(t) - X(t))] > 0, 



E[lim -p(t)X(t)} > 0. 

t— >oo 

Since 

p(t) = (X(t)X(t)) {j - 1) e 5 ( s+t \ 

we see that 

-p{t)X{t) = ^- 1 )(t)t^(()e , ( ,+, ). 

So, by considering 

- 1 _J_ S(s + t) 

X = -pi- 1 e t- 1 , 
x 

we try to let 

pT*(t) = X(t)Ke Bt , 
for some constants K and B.It is now clear that 

d(p^(t))^p^(t)^—(-pdt- { -^^dB(t)) 
7 — 1 a 



i 1 1 2-7(^-/9)2 

+ P~o T T 2 dt 

2 7 — 1 7 — 1 cH 

On the other hand we have that 

d(X(t)Ke Bt ) = BX{t)Ke Bt dt + X(t)Ke Bt [p + u(t)(n - p) - Ke Bt e^ 1 }dt 
+ X(t)Ke Bt cru(t)dB(t). 

Consider 

cr 2 (7-l) 
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K = e- m e~^^[B 



ip _ i Of - p? 

7-1 2 7 cr 2 ( 7 -l) 



2 J- 



For if to 6e independent oft, we must have B = — — ^-y, w/iic/i gives us 



K = 



With this K and 



we can conclude that we have 

It is now clear that 
which gives us that 



5 IP _ 1 (p - p f 
7-1 7-1 2 7 cr 2 ( 7 -l) 2 

0-2(7- 1) 
p^r(i) = X(t)Ke Bt . 
A(t) = K = A. 



p(i)-XXt) = X^/T^e 75 ' 
= XT(i)A 7 , 



so i/iat 



7.S(s + *) 



(p — P) 



-p(t)X(t) = -e"-^A 7 ^e 7/ ° tp ^T^iT ' ■ 1 



= -e, 



7<S(s+t) (p-p) 2 * Si (p-p) 2 j (p--p) o/ii 



75(s+t) ~ .2 (p-p) 2 „, (p-p) 



If 5,j, p deterministic, then 

E[TEip(t)(X(t) - X(t))] > -lime-^+^jx^e 

t— >oo 

= 0. 



2 (P--P) (P-P) p/jN. 



So we /lave £/ia£ i?[ lim — X(t))] = 0, which gives us that (A, u), where 

t— >oc 



A = 



IP 



1 (P P? 



:7~ 



and 



7-1 ' 7-1 2 ' cr 2 (7- l) 2 
0-2(7-1) 



is an optimal control. 

6. Necessary Maximum Principle 
To answer the question: if u is optimal does it satisfy 



E 



(28) 



max£ 
ueu 



H(t,X(t),u(t),p(t),q(t),f(t,-))\£ t 
H(t,X(t),u,p(t),q(t),r(t,-))\£ t 



we assume the following two requirements: 
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Al For all t, h such that <t <t + h <T, all i = 1, . . . , fc and for all bounded 
£ t -measurable a = a{u), the control f3(s) := (0, . . . , (3i(s), 0, . . . , 0) G U C 
R fe with 

f3{s) := ail [ttt+h] (s), 

belongs to As- 

A2 For all u, (3 G As with (3 bounded, there exists 5 > such that u + e(3 G Ag 
for all e G {-5,6). 

Given u, (3 G As with j3 bounded, define the process Y{t) = Y^ u ^\t) by 

Y{t) = ± X ^{t)\ e=0 = (Fi(i),...,y„(i)) T . 
Notice that Y(0) = and 

n n „ 

d^t) = A i (t)dt + 2^-(t)(iB J -(t)+2 / Cijfr^Njidz^t), 

3 = 1 J=l " /M o 

where 

Ai(t) = V x &j(t, X(t), u(t)) T y (t) + V u &j(t, X(t), u(t)) T /3(t), 
&(t) = ^x<y l3 {t,X{t),u{t)) T Y{t) + V u o t3 {t,X{t),u{t)) T p{t), 

Cij(t,z) = w x e ij {t,x{t),u{t)) T Y{t) + v u e tl (t,x(t),u(t)) 1 'p(t). 

We can then give a answer to the question. 

Theorem 6.1 (Partial Information Necessary Maximum Principle). Suppose u G 
As is a local maximum for J{u), meaning that for all bounded (3 G As there exists 
a 5 > such that u + e(3 G As for all e G {—5, 5) and 

h(e) := J{u + ef3),t G {-5,5) 

is maximal ate = 0. Suppose there exists a solution {p{t), q{t),r{t, z)) to the adjoint 
equation 

dp{t) = -V x H{t, X{t),u{t),p{t),q{t),r{t, -))dt + q{t)dB{t) 
+ / f{z,t)N{dz,dt), 

JRV: 



and 



< E 



lha[p{tY{X{t)-X{t))} 



< oo, 



for all u G As and p{t)Y{t, e) converges as t — > oo, uniformly in e, where Y{t, e) := 
d_X u+e P . Moreover assume that ifY{t) = Y^ u, ^{t), with corresponding coefficients 
K, Cij, we have 



E 



Y{t) T [qf{t)+ [ rr T {t,z)v(dx)]Y(t)dt 



< oo, 



and 



E 



/ p T {t)[i£ T {t,X{t),u{t))+ (( T {t,X{t),u(t),z)v{dz)]p{t)dt 
Jo Jm.a 



< 00. 
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Then u is a stationary point for E[H\£] in the sense that for all t > 0, 
E[V u H(t, X(t), u(t),p(t), q(t), f(t, -))\£ t ] = 0. 

Proof. Since 

< E 



lim[p(ty(X(t)-X(t))} 

t— >oo 



we have that 



E 



lim [p(i) T X" +e/3 (i)] 



for all f3 G As for some e. Define 



> E 



\im[p(tyX u (t))} 

t— too 



g(e) = lim [p(t) T X^ (t)}, 

t— >oo 



so that 



Eg(e) > Eg(0), 



for all (3 G As- This means that 



(Eg{e)) e=0 = 0. 



So 



9 



0=-(E lim WfX^it)] )| e=0 



= £ 
= £ 



-(hm[p(*) T X"+^)])| £=0 
ae t— >oo 

-^[pitfUx^m^] 

t— >oc oe 



the interchanging of the limit w.r.t. the derivative operator holds for uniform 
limits with uniform convergence of the derivative. Interchanging derivative and 
integration is justified if 



^(lim[p(i) T X^(t,a;)])| e= o 



for some integrable function F. Now let 

h(e) = J{u + e(3), 

so that we have 
= h'(0) 



E 



^°°{v :c /(t,XW,^)) T ^X il+ ^W| e= o + V tl /(t,XW^(t)) T /3(t)}dt 



+ ]im[p(ty-(X^(t))\e=o] 
t— too ae 
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Using Ito's Lemma we get 



E 



E 



{p(t) [V x 6(i, X(t),u(t)) T ^-X^(t)\ t=0 + V„6(t, X(t),u(t)) T (3(t) 



+ -X^(t)\ e=0 (-V x H(t, X(t),u(t),p(t),q(t), r(t, •)) 

+ q(t)(W x a(t, X(t), u{t)) T j^+^(t)\ t=0 + V u a(t, X(t), u(t)) J '/3(t) 

+ f(t,z)(V x e(t,X(t),u(t)) T j^+^{t)\ t=0 + V u 9(t,X(t),u(t)) T f3(t)Hdz)}dt 
Since 

V u H(t,x,u,p,q,r) = W u f(t,x,u) + V u b(t,x,u)p(t) + W u a(t,x,y)q(t) 
+ / V u 9(t,x,u,z)r(t,z)u(dz), 



and 



V u H(t,x,u,p,q,r) = V x f(t,x,u) + V x b(t,x,u)p(t) + V x a(t,x,y)q(t) 
+ / W x 9{t,x 1 u,z)r(t,z)v{dz), 



we have 
= E 



+ 

Define 
Then 



{ V u /(t, X(t), u(t)) + V u b(t, X{t),u{t))f + V u a(t, X(t), u(t))q 7 

fV u 9(t,X(t),u(t))/3(t))}dt 

/ W u H(t,X(t),u(t),p(t),q(t),f(t,-)) T /3(t)dt . 
Jo 

f3(s) := al [Lt+h] (s). 



E 



E 



rt+h 

J V u H(t, X(t),u(t),p(t), q(t), f(t, -)) T a(t)dt 



0. 



Differentiating with respect to ft at ft = gives 



E 



V u H{t, X{t),u(t),p{t),q{t), r(t, -)) T a 



Since this holds for all £ measurable a, we have that 



E 



V u H{t, X(t),ii(t),p(t),q(t),f{t, -)) T a\£ 



0. 



= 0, 



which proves the theorem. 



□ 
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